Implemented ordering for expanded iterators#8741
Conversation
github-actions
bot
added
python
JPPhoto
added
backend
Added a test for nested iterator execution ordering. (Failing at commit time!)
github-actions
bot
added
python
When a graph has nested iterators, some "ready to run" node combinations do not actually belong together. Previously, the scheduler would still try to build nodes for those mismatched combinations, which could cause the same work to run more than once. This change skips any combination that is missing a valid iterator parent, so nested iterator expansions run once per intended item.
JPPhoto
requested review from
blessedcoolant,
dunkeroni and
lstein
as code owners
lstein
left a comment
lstein
left a comment
There was a problem hiding this comment.
I'd like to be able to review this PR, but I've never quite understood how to use the collection/iteration functions in the workflow editor and don't think I have the expertise to identify edge cases and so forth.
If no one else is able to review, I'll come up to speed and give it a shot.
|
Following up on this, maybe you can provide an example workflow that illustrates the problems with the previous non-deterministic execution ordering and demonstrates how the PR fixes this? |
As it's non-deterministic, I can't guarantee that any example will show anything at all. The problem is that when iteration expands, prior to this change the resulting nodes are treated like any others and there are no guarantees that the collection of expanded nodes will execute in any order at all. After this change, the first iteration's nodes should always complete before the second iteration's start, the second's before the third's, etc. Hopefully this change accomplishes that; this section of code is getting very hard to read and understand and would likely benefit from a major refactor at some point. |
|
Drafting this as the collect node needs some adjustment even though execution seems to happen in order. |
|
Collect ordering is now resolved. Try the test workflow. |
lstein
left a comment
lstein
left a comment
There was a problem hiding this comment.
Consider using a @cached decorator on _get_iteration_path() as noted above.
There was a problem hiding this comment.
Pull request overview
This pull request implements best-effort in-order execution for nodes expanded by IterateInvocation to improve determinism and predictability in graph execution. The feature uses a cached "iteration path" (outer to inner indices for nested iterators) to order nodes in ready queues and collector inputs.
Changes:
- Added ordered execution for iterated nodes based on their iteration path, with lexicographic ordering by iteration indices
- Implemented sorted collection of items in
CollectInvocationbased on iteration context - Fixed iterator compatibility checking in node preparation to ensure correct iteration context matching
- Added comprehensive test for nested iterator ordering and relaxed test assertions for non-iterated items
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| tests/test_node_graph.py | Modified test assertion to use set comparison instead of ordered list, reflecting intentional lack of ordering guarantee for non-iterated parallel nodes |
| tests/test_graph_execution_state.py | Added new test validating deterministic ordering for nested iterators with parameterized execution to verify consistency |
| invokeai/app/services/shared/graph.py | Implemented core ordering functionality including iteration path calculation, ordered queue insertion, iterator compatibility checks, and sorted collector inputs |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Include iterator nodes in nx_graph_flat so iterators are prepared/expanded correctly. Fix connection type checks to allow subclass-to-base via issubclass. Harden iterator/collector validation to fail cleanly instead of crashing on missing edges. Remove unused nx_graph_with_data(). Added tests to verify proper functionality.
* Implemented ordering for expanded iterators * Update test_graph_execution_state.py Added a test for nested iterator execution ordering. (Failing at commit time!) * Filter invalid nested-iterator parent mappings in _prepare() When a graph has nested iterators, some "ready to run" node combinations do not actually belong together. Previously, the scheduler would still try to build nodes for those mismatched combinations, which could cause the same work to run more than once. This change skips any combination that is missing a valid iterator parent, so nested iterator expansions run once per intended item. * Fixed Collect node ordering * ruff * Removed ordering guarantees from test_node_graph.py * Fix iterator prep and type compatibility in graph execution Include iterator nodes in nx_graph_flat so iterators are prepared/expanded correctly. Fix connection type checks to allow subclass-to-base via issubclass. Harden iterator/collector validation to fail cleanly instead of crashing on missing edges. Remove unused nx_graph_with_data(). Added tests to verify proper functionality.
* Implemented ordering for expanded iterators * Update test_graph_execution_state.py Added a test for nested iterator execution ordering. (Failing at commit time!) * Filter invalid nested-iterator parent mappings in _prepare() When a graph has nested iterators, some "ready to run" node combinations do not actually belong together. Previously, the scheduler would still try to build nodes for those mismatched combinations, which could cause the same work to run more than once. This change skips any combination that is missing a valid iterator parent, so nested iterator expansions run once per intended item. * Fixed Collect node ordering * ruff * Removed ordering guarantees from test_node_graph.py * Fix iterator prep and type compatibility in graph execution Include iterator nodes in nx_graph_flat so iterators are prepared/expanded correctly. Fix connection type checks to allow subclass-to-base via issubclass. Harden iterator/collector validation to fail cleanly instead of crashing on missing edges. Remove unused nx_graph_with_data(). Added tests to verify proper functionality.
3 participants
Summary
Feature: best-effort in-order execution for nodes expanded by
IterateInvocation, subject to ready state.Why: iteration-expanded work could run in an arbitrary order when multiple iteration branches become ready at the same time, which reduces determinism and makes behavior harder to reason about.
How: compute a cached per-exec-node "iteration path" (outer to inner indices for nested iterators, respecting collector boundaries) and use it to ordered-insert ready nodes into the existing per-class ready queues. FIFO is preserved for equal iteration paths, and no blocking is introduced.
Related Issues / Discussions
QA Instructions
Create a collection, note the ordering, then iterate and pass values to a series of other nodes. Check that the outputs use the source collection's items in order.
Merge Plan
Checklist
What's Newcopy (if doing a release after this PR)